Speech recognition through disambiguation feedback

ABSTRACT

A request including audio data is received from a voice-enabled device. A string of phonemes present in the utterance is determined through speech recognition. At a later time, a subsequent user input corresponding to the request may be received, in which the user input is associated with one or more text keywords. The subsequent user input may be obtained in response to an active request. Alternatively, feedback may not be actively elicited, but rather collected passively. However it is obtained, the one or more keywords associated with the subsequent user input may be associated with the string of phonemes to indicate that the user is saying or mean those words when they product that string of phonemes. A user-specific speech recognition key for the user account is then updated to associate the string of phonemes with these words. A general speech recognition model can also be trained using the association.

BACKGROUND

Smart devices and digital assistants are able to help users with manytasks. In particular, voice-based control has become a popular andconvenient way of interacting with such devices. This allows users tointeract with the devices without having to hold or touch them or havingto navigate through a graphic interface. It is even possible for usersto online shop through voice interactions. For example, a user couldprovide a voice command such as “add diet cola to cart”, and such anitem would be added to their online shopping cart associated with thedevice. However, for a variety of reasons, the user's speech may not becorrectly understood.

BRIEF DESCRIPTION OF THE DRAWINGS

Various embodiments in accordance with the present disclosure will bedescribed with reference to the drawings, in which:

FIGS. 1A-1B illustrate an example scenario in which an automatic speechrecognition system has incorrectly interpreted a voice command, inaccordance with example embodiments.

FIGS. 2A-2C illustrate an example scenario in which a user is asked toprovide additional information (i.e., feedback) that helps clarify theirintent as well as improves speech recognition for future instances, inaccordance with example embodiments.

FIG. 3 illustrates an embodiment in which the additional information iselicited through a graphical interface displayed of display-baseddevice, in accordance with example embodiments.

FIG. 4 illustrates a diagrammatical representation of a voice-enabledapplication environment with user feedback learning, in accordance withexample embodiments.

FIG. 5 illustrates an example process of using user feedback to improvespeech recognition, in accordance with various embodiments.

FIG. 6 illustrates another example process of using user feedback toimprove speech recognition, in accordance with various embodiments.

FIG. 7 illustrates an example implementation device, in accordance withvarious embodiments of the present disclosure.

FIG. 8 illustrates an example implementation environment, in accordancewith various embodiments of the present disclosure.

DETAILED DESCRIPTION

In the following description, various embodiments will be described. Forpurposes of explanation, specific configurations and details are setforth in order to provide a thorough understanding of the embodiments.However, it will also be apparent to one skilled in the art that theembodiments may be practiced without the specific details. Furthermore,well-known features may be omitted or simplified in order not to obscurethe embodiment being described. Systems and methods in accordance withvarious embodiments of the present disclosure may overcome one or moreof the aforementioned and other deficiencies experienced in conventionalapproaches for voice control technology. In particular, variousembodiments are directed to collecting feedback to train models tobetter understand utterances and translate diverse user pronunciationsinto the intended words.

In various embodiments, a user may utter a voice directive that isdetected and picked up by a voice-enabled electronic device as an“utterance”. The utterance is then processed either on the device ortransmitted to a remote server for processing. Specifically, theutterance undergoes an automatic speech recognition (ASR) process,wherein the utterance text is determined. In some embodiment, it may bedetermined that the utterance is related to a certain type of request.This may be done by detecting for certain keywords, sentence structure,vocal tone, among other information provided in the utterance. Forexample, it may be determined that the utterance is related to shopping.Alternatively, the utterance may be determined to be an informationrequest, a device control input, among others. In some embodiments, theutterance may then be processed in the context of that type of request.This may include employing a natural language understanding (NLU) and/orname entity recognition (NER) framework specific to that type ofrequest. In doing so, an ASR error may be detected if one exists. ASRerror detection may be based on the confidence that the utterance wascorrectly understood. An ASR error may be detected if the confidencefalls below a predetermined threshold. An ARS error may also be detectedif there are multiple interpretations that meet the threshold. Forexample, the system may not be able to determine whether the user said“coke” or “coat”. A feedback prompt may be triggered upon detecting anASR error, in which the user is asked to provide additional informationto clarify what they meant. The feedback prompt may take one of variousforms. For example, the feedback prompt may be audio-based, in which theuser is asked via audio to provide additional voice input to supplementthe initial voice command. For example, the user could be asked torepeat their original command. In another example, the user may be askedto select between two possible interpretations. For example, the usermay be asked “did you mean ‘coke’ or ‘coat’?” The user may then respondto this prompt by voice, thereby providing feedback. In anotherembodiment, the feedback prompt may be presented in visual form, such asin the form of a message or in-app notification rather than throughaudio/voice. Either way, the collected additional user input can be usedto better understand the intent of the user's original voice command.

The additional user input may be associated with one or more knownwords. For example, the user input may be a selection between twoproducts. The name of the selected product is known and includes the oneor more keywords. Thus, the one or more keywords are associated with theoriginal utterance that could not be properly understood or causedambiguity. The association between the utterance and the one or morekeywords is then logged and used to update a user-specific speechrecognition key. The user-specific speech recognition key is specific toa particular user account, as it contains information on how tointerpret the way that user speaks and may not apply for other users. Insome embodiments, the association between the utterance and the one ormore keywords is also used to train a general speech recognition modelwhich is used to interpret utterances for a plurality of users. Theuser-specific speech recognition key may be updated upon receiving theadditional user input such that the next time the user makes the sameutterance, the user-specific speech recognition key is referenced, andthe utterance is interpreted as the one or more keywords associated withthe utterance. The general speech recognition model may be a statisticalmodel that is updated or trained in intervals and includes many entriesfrom different users.

FIG. 1A illustrates an example environment 100 wherein a user 102 isinteracting with a voice-enabled client device 104. A voice-enableclient device 104 includes, for example, any device having a microphoneor other component configured to generate audio data from sound in thefrequency range in which humans communicate. Although a voicecommunications device (e.g., an Amazon Echo) is illustrated, it shouldbe understood that the device 104 may be various other types ofelectronic devices that are capable of outputting audio and which haveaudio playback controls. These devices can include, for example,speakers, receivers, notebook computers, ultrabooks, personal dataassistants, video gaming consoles, televisions, set top boxes, smarttelevisions, portable media players, unmanned devices (e.g., drones orautonomous vehicles), wearable computers (e.g., smart watches, smartglasses, bracelets, etc.), display screens, display-less devices,virtual reality headsets, display-based devices, smart furniture, smarthousehold devices, smart vehicles, smart transportation devices, and/orsmart accessories, among others. In the example scenario 100 of FIG. 1A,the voice-enable client device 104 may also serve as an audio outputdevice such as for playing music. The voice-enable client device 104 cancommunicate with a server over at least one network, such as theInternet, a cellular network, a local area network (LAN), an Ethernet,Wi-Fi, or a dedicated network, among other such options.

As will be described further herein, the voice-enable client device 104may utilize a wakeword or other predetermined sound profile to activatesome functionality (e.g., send data to a remote device, such as audioinput data for speech analysis) as well as button-activated devices thatutilize a button (graphical, physical, or both) to enable someaudio-related functionality (e.g., a sound-capturing and sending mode).In this example, user 102 can speak a request within an environmentwhere the voice-enabled communications device 104 is located. Therequest may be any question, inquiry, instruction, phrase, or other setof one or more words/sounds. For example, the user may say, “Alexa, turnon kitchen lights” In this example, the word “Alexa” has a specialconnotation, in that it may be referred to as a wakeword, or activationword (the wakeword would be a different word, or a particular phoneme ora particular sound, such as the sound made by clapping or snapping yourfingers). In particular, a wakeword may be detected within audio inputdata detected by one or more microphones located on the voice-enabledcommunications device. Persons of ordinary skill in the art willrecognize, however, that the one or more microphones may alternativelybe located on a separate device in communication with the voice-enabledcommunications device. In some embodiments, after the wakeword isdetected, the voice-enabled communications device may begininterpreting/analyzing audio input data until no more speech isdetected.

In general, the voice-enabled communications device 104 constantlylistens for the wakeword and is otherwise inactive. Once the wakeword isrecognized, the voice-enabled communications device switches from apassive mode to an active mode. It should be noted that the wakeworddoes not have to be the first word or sound in a given sentence orrequest. The voice-enabled communications device can be configured suchthat it can record and store a limited amount of audio input data thatshould, in most instances, is the amount of time needed to speak asentence or more. Accordingly, even if a wakeword is recognized in themiddle or end of a sentence, the voice-enabled communications devicewill have retained the entire sentence which can then be analyzed bybackend servers to determine what is being requested.

An application executing on the voice-enabled communications device orotherwise in communication with the voice-enabled communications device,can analyze the user's speech that includes audio input data to performat least one function. The functions can include, for example, answeringquestions, playing music, reading audiobooks, controlling connecteddevices via voice commands/instructions, sending an electronic message,initiating a phone call, performing an online shopping action, amongother such functions.

In this example, the user 102 is utilizing an online shopping functionof the voice-enabled communication device 104. The voice-enable clientdevice 104 may be logged into an account on an e-commerce platformthrough which a user can purchase or otherwise select items from anelectronic catalog of items. The account may already be associated withinformation such as a payment method (e.g., credit card number),shipping address, billing address, and any other information needed tocomplete a transaction. Conventionally, the user would user a clientdevice with a display such as a personal computer or smart phone to logonto a website to access the e-commerce platform. The user can thenbrowse through the offerings or search using a keyword query to locatedproducts of interest. The user can perform various actions such asfinding out information about a product, adding a product to cart,removing a product from cart, checking out, and the like. Typicallyinformation is output to the user visually through the graphic interfacethat is displayed on the display of the device and user inputs areentered manually via a peripheral device such as a mouse or keyboard.With voice control technology, a user can do these or similar actionsthrough voice and audio communications, without the need for a devicewith a display or manually entered inputs. For example, and asillustrated, the user 102 may say “Alexa, add ACME brand detergent tocart”, as illustrated in quote bubble 106. Ideally, upon receiving thisvoice command, the command would be correctly interpreted and thecorrect product would be added to an electronic shopping cart associatedwith the user account. However, as illustrated in FIG. 1B, it may be thecase the user's command misunderstood. In this case, the spoke work“acme” is misunderstood through speech recognition as “acne”. Thee-commence application then used the term “acne detergent” to search fora product, and did not produce any qualifying search results. Thus, thevoice-enable client device 104 says “could not find acne detergent”, asindicated in quote bubble 108.

In some embodiments, the user account can be associated with a userprofile. The user profile may include information such as demographicinformation and previous behaviors. The previous behaviors may includemany types of information, such as product browsing history, purchasehistory, past utterances and associated actions and results, among otherinformation. It should be noted that other approaches can be implementedto login to a particular profile. For example, each profile may belogged into by, for example, saying the wakeword then a specialkeyword/phrase (e.g., sign in as Jane) and/or by biometrics (i.e.,speaker identification based on sound of voice and, if camera isavailable, facial recognition or, if fingerprint scanner, fingerprintID), among other such approaches.

In some embodiments, the contents of the audio input data areessentially streamed to a backend server (see FIG. 7 for furtherexplanation) such that at least a portion of the audio input data can bereceived by the backend server and analysis can begin on that portionand any subsequent portions immediately upon receipt. In particular, thebackend server can begin processing one or more portions of the audioinput data prior to the user having completed making the instructions.Thus, the backend server can start analyzing whatever portion of theaudio input data it received through a variety of techniques such asautomatic speech recognition (ASR) and natural language understanding(NLU) to convert the audio input data into a series of identifiablewords, and then to analyze those words in order to interpret the meaningof the request from the user. The backend server can utilize ASRtechniques to recognize the spoken words that were recorded and storedin an audio file and to translate them into known text that can then beanalyzed by NLU techniques to attempt to decipher the meaning of therequest from user. Any suitable computer implemented speech-to-texttechnique may be used to convert the received audio signal(s) into text,such as SOFTSOUND speech processing technologies available from theAutonomy Corporation, which is headquartered in Cambridge, England,United Kingdom. In some embodiments, one or more filters may be appliedto the received audio input data to reduce or minimize extraneous noise,however this is not required. In this example, analyzing the audio inputdata can include determining a product “ACME brand detergent” anintended action “Add to cart”. The backend server can then identify theproduct from the electronic catalog and add it to the electronic cartassociated with the user account. In some embodiments, the device 104may provide a confirmation such as “ACME brand detergent added to cart”.In this case, the electronic catalog of products may only contain oneproduct that is responsive to the product query “ACME brand detergent”.

In an ideal scenario, the system can confidently determine what actionto take based on the user's voice command. However, the user's utterancemay not be properly interpreted through automatic speech recognition(ASR) and natural language understanding (NLU). Thus, the system may beunable to determine what words are being spoken chose the wrong words.E-commerce technology is provided as an example application of thepresent techniques. However, the techniques described herein can be usedto improve various other technologies such as for example, answeringquestions, playing music, reading audiobooks, controlling connecteddevices via voice commands/instructions, sending an electronic message,initiating a phone call, among other such technologies. As used herein,the term “automatic speech recognition” or “ASR” refers to and includesvarious types and stages of speech recognition, natural languageunderstanding, named entity recognition and understanding, categoricaland contextual understanding, user intent understanding (e.g., howspoken language translates into actual user intent), and the like.

FIGS. 2A-2C illustrates such a scenario in which a user 202 is asked toprovide additional information (i.e., feedback) that helps clarify theirintent as well as improves speech recognition for future instances. InFIG. 2A, a user provides a voice command by saying, for example, “Alexa,add ACME brand detergent to cart”, as indicated in quote bubble 206. Thevoice command is captured by a voice-enabled client device 204 and sentto a server for processing. In some embodiments, the processing andother techniques described herein may be performed completely or in partonboard the voice-enabled client device 204. The voice command isreceived as audio data which is processed through automatic speechrecognition techniques. In this example, an error occurs during thespeech recognition, in which the speech recognition cannot distinguishbetween whether the user said “acme” or “acne”. As illustrated in FIG.2B, this error causes the voice-enabled client device 204 to ask theuser to select which one they meant. For example, the voice-enabledclient device may say “Please say one if you meant acme or say two ifyou meant acne”, as indicated in quote bubble 208. As illustrated inFIG. 2B, the user 202 may then respond directly to the question bysaying “one”, as indicated in quote bubble 210. Thus, it can bedetermined from this answer that the user originally said “acme” not“acne”.

FIG. 3 illustrates an embodiment 300 in which the additional informationis elicited through a graphical interface displayed of display-baseddevice 302. The display-based device may be a personal computer, asmartphone, a table, or the like. Specifically, a prompt 306 may bedisplayed on the display 304 of the device 302. In some embodiments, theprompt 306 may appear as a pop-up as a part of the graphic interface ofthe e-commerce platform accessed through the device 302. The prompt 306may be any type of graphical element that can be displayed on thedisplay 304 of the device. In this example, as illustrated in FIG. 3 ,the user may be prompted through the graphic interface to select betweena first button 308 a representing “acme” and a second button 308 brepresenting “acne”. The feedback collected through any of theabove-described means can be used to complete the initial request. Thisfeedback can also be used to train the speech recognition to betterunderstand that user and other users in the future, as described indetail below.

FIG. 4 illustrates a diagrammatical representation 400 of avoice-enabled application environment 408 with user feedback learning,in accordance with example embodiments. A voice-enabled client device402 captures an utterance spoken by a user, such as a command followinga wakeword. The voice-enabled client device 402 then sends the audiodata representative of the utterance to a server-side voice-enabledapplication environment 408 over a network 406. The voice-enabled clientdevice 402 may be any type of client device that includes an audiooutput device such as a speaker and an audio input device such asmicrophone, and network connectivity. This includes specialvoice-communication only devices, personal computers, tablet computers,smart phones, notebook computers, and the like. The network 406 caninclude any appropriate network, such as the Internet, a local areanetwork (LAN), a cellular network, an Ethernet, Wi-Fi, Bluetooth,radiofrequency, or other such wired and/or wireless network. Thevoice-enabled application environment 408 can include any appropriateresources for performing the various functions described herein, and mayinclude various servers, data stores, and other such components known orused for providing content from across a network (or from the cloud).The voice-enabled client device 402 may be logged into a user accountprovided by an application platform 420 such as an online store.

The audio data from the voice-enabled client device 408 is receivedthrough an interface 410, which facilitates communications between thevoice-enabled client device 402 and the voice-enabled applicationenvironment 408. For example, the interface 410 may process datareceived over the network 406 for use by other components of theserver-side voice-enabled application environment 408. For example, insome embodiments, the interface 410 may perform some signal processingon the received audio data such as noise reduction, filtering, and thelike. The interface 410 may also prepare data to be transmitted over thenetwork 406 to the voice-enabled client device 402. In this example, theaudio data, or a processed version of the audio data is further analyzedby an automatic speech recognition (ASR) and/or natural languageunderstanding (NLU) engine 412, which applies a variety of techniques toconvert the audio input data into a series of identifiable words, andthen to analyze those words in order to interpret the meaning of therequest from the user. The backend server can utilize ASR techniques torecognize the spoken words that were recorded and stored in the audiodata and to translate them into known text that can then be analyzed byNLU techniques to attempt to decipher the meaning of the request fromuser.

In some embodiments, automatic speech recognition may be performed onthe audio data to determine a string of phonemes present in theutterance. The automatic speech recognition also attempts to translatethe string of phonemes into one or more words. The automatic speechrecognition may be associated with a list of possible words. This mayinclude common words of a particular language as well as names andmade-up words. In some embodiments, the list of possible words isspecific to a certain application. For example, for an e-commerceapplication, the list of words may include various name brands ande-commerce specific terms. The ASR/NLU engine(s) 412 may access auser-specific speech recognition key 418 associated with the useraccount and/or a general speech recognition model 414 to perform speechrecognition on the string of phonemes. The user-specific speechrecognition key 418 is a rules-based or deterministic means of speechrecognition. The general speech recognition model 414 can also betrained using an association of the string of phonemes and the one ormore keywords. The general speech recognition model is a statisticalmodel that predicts the most likely words for a given phoneme stringbased on training data that includes many examples of how differentusers pronounce different words. In some embodiments, the user-specificspeech recognition key 418 is referenced for the specific user account,and the general speech recognition model 414 is referenced during speechrecognition for a plurality of user accounts. In various embodiments,the ASR/NLU engine(s) may perform various types and stages of speechrecognition, natural language understanding, named entity recognitionand understanding, categorical and contextual understanding, user intentunderstanding (e.g., how spoken language translates into actual userintent), and the like.

A user feedback service 416 may actively or passing collect feedbackfrom users to better understand what users are saying and therebyimprove the ASR/NLU engine 412. For example, actively collectingfeedback may occur when an error is detected during the automatic speechrecognition. For example, errors may include being unable to determineany words that correspond to the string of phonemes or being unable todistinguish between multiple words that seem to correspond to the stringof phonemes. If there is an error detected in the automatic speechrecognition process, additional information may be requested from theuser. This may be done via the voice-enabled client device from whichthe initial request was received, or via a display-based client device.For example, the voice-enabled device may say “Please say one if youmeant coke or say two if you meant coat”. The user may then responddirectly to the question with their answer through voice, just like areal-time dialogue. In some embodiments, the user may provide theiranswer directly without a wakeword. Alternatively, a prompt may begenerated in the graphic interface of the e-commerce platform displayedon a client device such as a computer or smartphone. For example, theuser may be prompted through the graphic interface to select between theoptions “coat” or “coke”.

An example of passively collecting feedback is when there is no errordetected in the automatic speech recognition process, and the systemthinks it has successfully determined the words and intention of theuser. Depending on the application, an action may be performed based onthat determination. For example, in an e-commerce application, aspecific product may be added to a cart. However, it may be detectedthat the user later edit or reversed that action. This can serve aspassive feedback that the utterance may not have been correctlyinterpreted. Additionally, the user's edit may provide information thatcan help correct the interpretation. For example, if the user replacesthe item that was added to the cart with another item, it may be learnedthat the utterance or string of phonemes corresponds to one or moreassociated with the item that the user manually added in place of theautomatically added item. In some embodiments, this information resideson the application platform (e.g., collection of data stores that hostthe application), and the feedback service 416 can access theapplication platform to collect this information.

Whether active or passively collected, the user feedback includes one ormore known words that correspond to the string of phonemes. Thus, thestring of phonemes is associated with those words, indicating that whena user is saying those words when they produce such a string ofphonemes. In some embodiments, the user-specific speech recognition key418 for that user account is then updated to associate the string ofphonemes with these one or more known words. Similarly, the generalspeech recognition model 414 can also be trained using this associationas a piece of training data. In some embodiments, the user-specificspeech recognition key 418 is updated faster than the general speechrecognition model 414, such as upon receiving the feedback. In someembodiments, the user account associated with the initial request mayhave various demographic data, such as language, geographic region, age,gender, and the like. The general speech recognition model may betrained using an association of the string of phonemes and the one ormore text keywords and the demographic data.

FIG. 5 illustrates an example process 500 of using user feedback toimprove speech recognition, in accordance with various embodiments. Itshould be understood that, for any process discussed herein, there canbe additional, fewer, or alternative steps performed in similar oralternative orders, or in parallel, within the scope of the variousembodiments. In this example, a request including audio data is received502 from a voice-enabled client device. The audio data is representativeof an utterance captured by the device. In some embodiments, the deviceassociated with a user account. For example, the user account may bethat of an e-commerce platform that has a database of productsavailable. Automatic speech recognition may be performed on to audiodata to determine 504 a string of phonemes present in the utterance. Theautomatic speech recognition also attempts 506 to translate the stringof phonemes into one or more known words. The automatic speechrecognition may be associated with a list of possible words. This mayinclude common words of a particular language as well as names andmade-up words. In some embodiments, the list of possible words isspecific to a certain application. For example, for a shoppingapplication, the list of words may include various name brands andshopping-specific terms.

An error may occur during the automatic speech recognition. For example,errors may include being unable to determine any words that correspondto the string of phonemes or being unable to distinguish betweenmultiple words that seem to correspond to the string of phonemes. Insome embodiments, confidence scores may be used as a measure todetermine whether a word corresponds to a string of phonemes. It isdetermined 508 if an error is detected in the automatic speechrecognition. In some embodiments, if there is no error detected in theautomatic speech recognition process, then utterance text issuccessfully determined 510. Depending on the application, variousthings can then be done using the text to fulfill the user's request. Ifthere is an error detected in the automatic speech recognition process,additional information may be requested 512 from the user. This may bedone via the voice-enabled client device from which the initial requestwas received, or via a display-based client device. For example, thevoice-enabled device may say “Please say one if you meant coke or saytwo if you meant coat”. The user may then respond directly to thequestion with their answer through voice, just like a real-timedialogue. In some embodiments, the user may provide their answerdirectly without a wakeword. Alternatively, a prompt may be generated inthe graphic interface of the e-commerce platform displayed on a clientdevice such as a computer or smartphone. For example, the user may beprompted through the graphic interface to select between the options“coat” or “coke”.

The additional information is then received 514. Regardless of the meansof obtaining the additional information, the additional informationincludes one or more known words. Thus the one or more keywords areassociated with the string of phonemes obtained from the original voicedata. Essentially, the speech recognition system now knows that thestring of phonemes corresponds to these keywords. A user-specific speechrecognition key for the user account is then updated 518 to associatethe string of phonemes with these one or more keywords. A general speechrecognition model can also be trained 520 using an association of thestring of phonemes and the one or more keywords. In some embodiments,the user-specific speech recognition key is referenced for the specificuser account, and the general speech recognition model is referencedduring speech recognition for a plurality of user accounts. In someembodiments, the user-specific speech recognition key is updated fasterthan the general speech recognition model, such upon receiving theadditional information. In some embodiments, the user account associatedwith the initial request may have various demographic data, such aslanguage, geographic region, age, gender, and the like. The generalspeech recognition model may be trained using an association of thestring of phonemes and the one or more text keywords and the demographicdata. In some embodiments, the general speech recognition model is apart of at least one of an automatic speech recognition (ASR) model, anatural language understanding (NLU) model, or a named entityrecognition (NER) model associated with an e-commerce platform.

FIG. 6 illustrates another example process 600 of using user feedback toimprove speech recognition, in accordance with various embodiments. Inthis example, a request including audio data is received 602 from avoice-enabled client device. The audio data is representative of anutterance captured by the device. In some embodiments, the deviceassociated with a user account. A string of phonemes present in theutterance is determined 604, such as through an automatic speechrecognition process. At a later time, a subsequent user inputcorresponding to the request may be received 606, in which the userinput is associated with one or more text keywords.

The subsequent user input may be obtained in response to an activerequest for feedback from the user to disambiguate between a pluralityof possible items or words. In this case, an error may have beendetected and thus feedback from the user is actively elicited. In someembodiments, requesting additional information is performed over voicethough a voice-enabled client device. In some embodiments, requestingadditional information is performed via graphical interface on adisplay-based client device. Alternatively, feedback may not be activelyelicited, but rather collected passively. For example, the speechrecognition may think it successfully recognized the utterance and adownstream system performs an action accordingly. However, this actionmay not be what the user intended because the utterance was actually notcorrectly recognized. The user may correct this, such as by takin anitem out of the cart manually and replacing it with a different item.This may provide the additional information.

However it is obtained, the one or more keywords associated with thesubsequent user input may be associated 608 with the string of phonemesto indicate that the user is saying or mean those keywords when theyproduct that string of phonemes. A user-specific speech recognition keyfor the user account is then updated 610 to associate the string ofphonemes with these one or more keywords. The user-specific speechrecognition key is a rules-based or deterministic means of speechrecognition. A general speech recognition model can also be trained 612using an association of the string of phonemes and the one or morekeywords. In some embodiments, the general speech recognition may betrained many different examples of phoneme strings that are associatedwith a certain one or more keywords. The general speech recognitionmodel is a statistical model that predicts the most likely words for agiven phoneme string based on training data.

In some embodiments, the user-specific speech recognition key isreferenced for the specific user account, and the general speechrecognition model is referenced during speech recognition for aplurality of user accounts. In some embodiments, the user-specificspeech recognition key is updated faster than the general speechrecognition model, such upon receiving the additional information. Insome embodiments, the user account associated with the initial requestmay have various demographic data, such as language, geographic region,age, gender, and the like. The general speech recognition model may betrained using an association of the string of phonemes and the one ormore text keywords and the demographic data. In some embodiments, thegeneral speech recognition model is a part of at least one of anautomatic speech recognition (ASR) model, a natural languageunderstanding (NLU) model, or a named entity recognition (NER) modelassociated with an e-commerce platform.

In some embodiments, after the user-specific speech recognition key isupdated, it can be used to translate the string of phonemes into thecorrect text keywords. For example, if a second request is received fromthe same user account and that include the string of phonemes, it can bedetermined through referencing the user-specific speech recognition keythat the user is referring to those words. In some embodiments, afterthe general speech recognition model is trained using the associationbetween the string of phonemes and the one or more keywords, and arequest from a different user includes that string of phonemes, thegeneral speech recognition model may be used to recognize the string ofphonemes as referring to those words even for a different user account.

FIG. 7 is another example environment 700 for implementing aspects inaccordance with various embodiments. In this example, voice-enabledcommunications device 104, in some embodiments, may correspond to anytype of electronic device capable of being activated in response todetecting a specific sound. Voice-enabled communications device 104 may,in some embodiments, after detecting the specific sound (e.g., awakeword), recognize commands (e.g., audio commands, inputs) withincaptured audio, and may perform one or more actions in response to thereceived commands. Various types of electronic devices may include, butare not limited to, notebook computers, ultrabooks, tablet computers,mobile phones, smart phones, personal data assistants, video gamingconsoles, televisions, set top boxes, smart televisions, portable mediaplayers, and wearable computers (e.g., smart watches, smart glasses,bracelets, etc.), display screens, displayless devices (e.g., AmazonEcho), other types of display-enabled devices, smart furniture, smarthousehold devices, smart vehicles, smart transportation devices, and/orsmart accessories, among others. In some embodiments, voice-enabledcommunications device 104 may be relatively simple or basic in structuresuch that no mechanical input option(s) (e.g., keyboard, mouse,trackpad) or touch input(s) (e.g., touchscreen, buttons) may beprovided. For example, voice-enabled communications device 104 may becapable of receiving and outputting audio, and may include power,processing capabilities, storage/memory capabilities, and communicationcapabilities. Voice-enabled communications device 104 may include aminimal number of input mechanisms, such as a power on/off switch,however primary functionality, in one embodiment, of voice-enabledcommunications device 104 may solely be through audio input and audiooutput. For example, voice-enabled communications device 104 may listenfor a wakeword by continually monitoring local audio. In response to thewakeword being detected, voice-enabled communications device 104 mayestablish a connection with backend server, send audio input data tobackend server, and await/receive a response from backend server. Insome embodiments, however, non-voice-enabled devices may alsocommunicate with backend server (e.g., push-to-talk devices).Voice-enabled communications device 104 may include one or moreprocessors 702, storage/memory 704, communications circuitry 706, one ormore microphones 708 or other audio input devices (e.g., transducers),one or more speakers 710 or other audio output devices, as well as anoptional visual input/output (“I/O”) interface 712. However, one or moreadditional components may be included within voice-enabledcommunications device 104, and/or one or more components may be omitted.For example, voice-enabled communications device 104 may include a powersupply or a bus connector. As another example, voice-enabledcommunications device 104 may not include a visual I/O interface.Furthermore, while multiple instances of one or more components may beincluded within voice-enabled communications device 104, for simplicityonly one of each component has been shown. Processor(s) 702 may includeany suitable processing circuitry capable of controlling operations andfunctionality of voice-enabled communications device 104, as well asfacilitating communications between various components withinvoice-enabled communications device 104. In some embodiments,processor(s) 702 may include a central processing unit (“CPU”), agraphic processing unit (“GPU”), one or more microprocessors, a digitalsignal processor, or any other type of processor, or any combinationthereof. In some embodiments, the functionality of processor(s) 702 maybe performed by one or more hardware logic components including, but notlimited to, field-programmable gate arrays (“FPGA”), applicationspecific integrated circuits (“ASICs”), application-specific standardproducts (“ASSPs”), system-on-chip systems (“SOCs”), and/or complexprogrammable logic devices (“CPLDs”). Furthermore, each of processor(s)702 may include its own local memory, which may store program modules,program data, and/or one or more operating systems. However,processor(s) 702 may run an operating system (“OS”) for voice-enabledcommunications device 104, and/or one or more firmware applications,media applications, and/or applications resident thereon. Storage/memory704 may include one or more types of storage mediums such as anyvolatile or non-volatile memory, or any removable or non-removablememory implemented in any suitable manner to store data on voice-enabledcommunications device 104. For example, information may be stored usingcomputer-readable instructions, data structures, and/or program modules.Various types of storage/memory may include, but are not limited to,hard drives, solid state drives, flash memory, permanent memory (e.g.,ROM), electronically erasable programmable read-only memory (“EEPROM”),CD ROM, digital versatile disk (“DVD”) or other optical storage medium,magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, RAID storage systems, or any other storagetype, or any combination thereof. Furthermore, storage/memory 704 may beimplemented as computer-readable storage media (“CRSM”), which may beany available physical media accessible by processor(s) 702 to executeone or more instructions stored within storage/memory 704. In someembodiments, one or more applications (e.g., gaming, music, video,calendars, lists, etc.) may be run by processor(s) 702, and may bestored in memory 704. In some embodiments, storage/memory 704 mayinclude one or more modules and/or databases, such as speech recognitionmodule 703, list of wakewords database 716, and wakeword detectionmodule 718. Speech recognition module 703 may, for example, include anautomatic speech recognition (“ASR”) component that recognizes humanspeech in detected audio. Speech recognition module 703 may also includea natural language understanding (“NLU”) component that determines userintent based on the detected audio. Also included within speechrecognition module 703 may be a text-to-speech (“TTS”) component capableof converting text to speech to be outputted by speaker(s) 710, and/or aspeech-to-text (“STT”) component capable of converting received audiosignals into text to be sent to backend server 708 for processing. Listof wakewords database 716 may be a database stored locally onvoice-enabled communications device 104 that includes a list of acurrent wakeword for voice-enabled communications device 104, as well asone or more previously used, or alternative, wakewords for voice-enabledcommunications device. In some embodiments, user 104 may set or programa wakeword for voice-enabled communications device 104. The wakeword maybe programmed directly on voice-enabled communications device 104, or awakeword or words may be set by the individual via a backend serverapplication (app) that is in communication with backend server 708. Forexample, a user may use their mobile device having the backend serverapplication running thereon to set the wakeword. The specific wakewordmay then be communicated from the mobile device to backend server 708,which in turn may send/notify voice-enabled communications device 104 ofthe individual's selection for the wakeword. The selected activation maythen be stored in list of wakewords database 716 of storage/memory 704.Wakeword detection module 718 may include an expression detector thatanalyzes an audio signal produced by microphone(s) 708 to detect awakeword, which generally may be a predefined word, phrase, or any othersound, or any series of temporally related sounds. Such an expressiondetector may be implemented using keyword spotting technology, as anexample. A keyword spotter is a functional component or algorithm thatevaluates an audio signal to detect the presence of a predefined word orexpression within the audio signal detected by microphone(s) 708. Ratherthan producing a transcription of words of the speech, a keyword spottergenerates a true/false output (e.g., a logical I/O) to indicate whetheror not the predefined word or expression was represented in the audiosignal. In some embodiments, an expression detector may be configured toanalyze the audio signal to produce a score indicating a likelihood thatthe wakeword is represented within the audio signal detected bymicrophone(s) 708. The expression detector may then compare that scoreto a threshold to determine whether the wakeword will be declared ashaving been spoken. In some embodiments, a keyword spotter may be usesimplified ASR techniques. For example, an expression detector may use aHidden Markov Model (“HMM”) recognizer that performs acoustic modelingof the audio signal and compares the HMM model of the audio signal toone or more reference HMM models that have been created by training forspecific trigger expressions. An HMM model represents a word as a seriesof states. Generally a portion of an audio signal is analyzed bycomparing its HMM model to an HMM model of the trigger expression,yielding a feature score that represents the similarity of the audiosignal model to the trigger expression model. In practice, an HMMrecognizer may produce multiple feature scores, corresponding todifferent features of the HMM models. An expression detector may use asupport vector machine (“SVM”) classifier that receives the one or morefeature scores produced by the HMM recognizer. The SVM classifierproduces a confidence score indicating the likelihood that an audiosignal contains the trigger expression. The confidence score is comparedto a confidence threshold to make a final decision regarding whether aparticular portion of the audio signal represents an utterance of thetrigger expression (e.g., wakeword). Upon declaring that the audiosignal represents an utterance of the trigger expression, voice-enabledcommunications device 104 may then begin sending the audio signal tobackend server 708 for detecting and responds to subsequent utterancesmade by a user. Communications circuitry 706 may include any circuitryallowing or enabling voice-enabled communications device 104 tocommunicate with one or more devices, servers, and/or systems. Forexample, communications circuitry 706 may facilitate communicationsbetween voice-enabled communications device 104 and backend server 708.Communications circuitry 706 may use any communications protocol, suchas any of the previously mentioned exemplary communications protocols.In some embodiments, voice-enabled communications device 104 may includean antenna to facilitate wireless communications with a network usingvarious wireless technologies (e.g., Wi-Fi, Bluetooth, radiofrequency,etc.). In yet another embodiment, voice-enabled communications device104 may include one or more universal serial bus (“USB”) ports, one ormore Ethernet or broadband ports, and/or any other type of hardwireaccess port so that communications circuitry 706 allows voice-enabledcommunications device 104 to communicate with one or more communicationsnetworks. Voice-enabled communications device 104 may also include oneor more microphones 708 and/or transducers. Microphone(s) 708 may be anysuitable component capable of detecting audio signals. For example,microphone(s) 708 may include one or more sensors for generatingelectrical signals and circuitry capable of processing the generatedelectrical signals. In some embodiments, microphone(s) 708 may includemultiple microphones capable of detecting various frequency levels. Asan illustrative example, voice-enabled communications device 104 mayinclude multiple microphones (e.g., four, seven, ten, etc.) placed atvarious positions about voice-enabled communications device 104 tomonitor/capture any audio outputted in the environment wherevoice-enabled communications device 104 is located. The variousmicrophones 708 may include some microphones optimized for distantsounds, while some microphones may be optimized for sounds occurringwithin a close range of voice-enabled communications device 104.Voice-enabled communications device 104 may further include one or morespeakers 710. Speaker(s) 710 may correspond to any suitable mechanismfor outputting audio signals. For example, speaker(s) 710 may includeone or more speaker units, transducers, arrays of speakers, and/orarrays of transducers that may be capable of broadcasting audio signalsand or audio content to a surrounding area where voice-enabledcommunications device 104 may be located. In some embodiments,speaker(s) 710 may include headphones or ear buds, which may bewirelessly wired, or hard-wired, to voice-enabled communications device104, that may be capable of broadcasting audio. In some embodiments, oneor more microphones 708 may serve as input devices to receive audioinputs, such as speech. Voice-enabled communications device 104, maythen also include one or more speakers 710 to output audible responses.In this manner, voice-enabled communications device 104 may functionsolely through speech or audio, without the use or need for any inputmechanisms or displays. In one exemplary embodiment, voice-enabledcommunications device 104 includes I/O interface 712. The input portionof I/O interface 712 may correspond to any suitable mechanism forreceiving inputs from a user of voice-enabled communications device 104.For example, a camera, keyboard, mouse, joystick, or external controllermay be used as an input mechanism for I/O interface 712. The outputportion of I/O interface 712 may correspond to any suitable mechanismfor generating outputs from voice-enabled communications device 104. Forexample, one or more displays may be used as an output mechanism for I/Ointerface 712. As another example, one or more lights, light emittingdiodes (“LEDs”), or other visual indicator(s) may be used to outputsignals via I/O interface 712 of voice-enabled communications device104. In some embodiments, one or more vibrating mechanisms or otherhaptic features may be included with I/O interface 712 to provide ahaptic response to user 104 from voice-enabled communications device104. Persons of ordinary skill in the art will recognize that, in someembodiments, one or more features of I/O interface 712 may be includedin a purely voice-enabled version of voice communications device 104.For example, one or more LED lights may be included on voice-enabledcommunications device 104 such that, when microphone(s) 708 receiveaudio from user 104, the one or more LED lights become illuminatedsignifying that audio has been received by voice-enabled communicationsdevice 104. In some embodiments, I/O interface 712 may include a displayscreen and/or touch screen, which may be any size and/or shape and maybe located at any portion of voice-enabled communications device 104.Various types of displays may include, but are not limited to, liquidcrystal displays (“LCD”), monochrome displays, color graphics adapter(“CGA”) displays, enhanced graphics adapter (“EGA”) displays, variablegraphics array (“VGA”) display, or any other type of display, or anycombination thereof. Still further, a touch screen may, in someembodiments, correspond to a display screen including capacitive sensingpanels capable of recognizing touch inputs thereon. FIG. 7 also includesbackend server 766, as mentioned previously, which may be incommunication with voice-enabled communications device 104. Backendserver 766 (e.g., part of a resource provider environment) includesvarious components and modules including, but not limited to, automaticspeech recognition (“ASR”) module 758 (which may include, for example,speech-to-text (“STT”) functionality), natural language understanding(“NLU”) module 760, applications module 762, and text-to-speech (“TTS”)module 764. In some embodiments, backend server 766 may also includecomputer readable media, including, but not limited to, flash memory,random access memory (“RAM”), and/or read-only memory (“ROM”). Backendserver 766 may also include various modules that store software,hardware, logic, instructions, and/or commands, such as, a speakeridentification (“ID”) module, a user profile module, or any othermodule, or any combination thereof. The speech-to-text functionality andtext-to-speech functionality may be combined into a single modulecapable of performing both STT and TTS processing, or separate TTS andSTT modules may, alternatively, be used. ASR module 758 may beconfigured such that it recognizes human speech in detected audio, suchas audio captured by voice-enabled communications device 104, which isthen sent to backend server 766. ASR module 758 may include, in oneembodiment, one or more processor(s) 752, storage/memory 754, andcommunications circuitry 756. Processor(s) 752, storage/memory 754, andcommunications circuitry 756 may, in some embodiments, be substantiallysimilar to processor(s) 702, storage/memory 704, and communicationscircuitry 706, which are described in greater detail above, and theaforementioned descriptions of the latter may apply. NLU module 760 maybe configured such that it determines user intent based on the detectedaudio received from voice-enabled communications device 104. NLU module760 may include processor(s) 752, storage/memory 754, and communicationscircuitry 756. Applications module 762 may, for example, correspond tovarious action specific applications or servers capable of processingvarious task specific actions. Applications module 762 may furthercorrespond to first party applications and/or third party applicationsoperable to perform different tasks or actions. For example, based onthe context of audio received from voice-enabled communications device104, backend server 766 may use a certain application to perform anaction, such refining an active play queue of media content.Applications module 762 may include processor(s) 752, storage/memory754, and communications circuitry 756. As an illustrative example,applications module 762 may correspond to a media service. Theelectronic media service application of the applications module 762 canbe associated with a customer account. The customer account can includeat least one profile stored in, for example, user information that canbe linked to the electronic media service application in applicationsmodule 762. Audio input data can be received at automatic speechrecognition module 758 from voice communications device 104. Theautomatic speech recognition module 758 can use automatic speechrecognition (ASR) techniques on the audio input data to generate textdata of the audio input data. The natural language understanding module760 can use natural language understanding (NLU) techniques on the textdata to determine refinement/attribute information to manage the activeplay queue. The electronic media service application of the applicationsmodule 762 can receive information that can be used to refine orotherwise control the playback of media content, where refining theplayback of media content can include filtering media content from anactive play queue of media content, adding media content to the activeplay queue of media content, re-ordering the sequence of content in theplay-queue, supplementing the active play queue, and/or changing thefrequency of playback of content in the play-queue. In accordance withan embodiment, the application can determine whether there is an activeplay queue of media content configured to play on the voicecommunications device, such as a playlist of music, a station of music,a mix of songs, etc. In the situation where there is no media contentbeing played by the voice communications device or no active play queueof media content, the electronic media service application determinesmedia content using information in the request. The information can beused to search a catalog of media content to identify media content inresponse to the spoken question or request. For example, the informationcan be used to identify media content associated with a mood, a tempo, agenre, an artist, a year, a decade, an activity as well as any othertopic or interest. The identified media can thereafter be played usingthe voice communications device. In the situation where there is anactive play queue of media content, the information can be used torefine the play queue. For example, the information can includeinstructions such as refinement instructions that can be used to filterthe play queue and/or add media content to the play queue from a catalogof media content. In various embodiments, the user can further refinethe playback of media content. For example, in the situation where theuser is engaging in a multi-turn dialog interaction with the voicecommunications device, where the user sends multiple requests to thevoice communications device to refine the media playing, the user canfirst instruct the device to play “happy” music. If the user desires“happier” music, the user can instruct the voice communications deviceto play “happier” music. TTS module 764 may employ varioustext-to-speech techniques. It should be noted that techniques for takingtext and converting it into audio input data that can represent speechare well known in the art and need not be described in further detailherein, any suitable computer implemented techniques may be used. TTSmodule 764 may also include processor(s) 752, storage/memory 754, andcommunications circuitry 756. Persons of ordinary skill in the art willrecognize that although each of ASR module 758, NLU module 760,applications module 762, and TTS module 764 include instances ofprocessor(s) 752, storage/memory 754, and communications circuitry 756,those instances of processor(s) 752, storage/memory 754, andcommunications circuitry 756 within each of ASR module 758, NLU module760, applications module 762, and STT/TTS module 764 may differ. Forexample, the structure, function, and style of processor(s) 752 withinASR module 758 may be substantially similar to the structure, function,and style of processor(s) 752 within NLU module 760, however the actualprocessor(s) 752 need not be the same entity.

In accordance with various embodiments, different approaches can beimplemented in various environments in accordance with the describedembodiments. For example, FIG. 8 illustrates an example of anenvironment 800 for implementing aspects in accordance with variousembodiments (e.g., a resource provider environment). As will beappreciated, although a Web-based environment is used for purposes ofexplanation, different environments may be used, as appropriate, toimplement various embodiments. The system includes voice communicationsdevice 104, which can include any appropriate device operable to sendand receive requests, messages or information over network 804 andconvey information back to an appropriate device. The network caninclude any appropriate network, including a telephone network providedby a telecommunication operator, an intranet, the Internet, a cellularnetwork, a local area network, wireless network, or any other suchnetwork or combination thereof. Communication over the network can beenabled via wired or wireless connections and combinations thereof. Inthis example, the network includes the Internet, as the environmentincludes a Web server 806 for receiving requests and serving content inresponse thereto, although for other networks, an alternative deviceserving a similar purpose could be used, as would be apparent to one ofordinary skill in the art. The illustrative environment includes atleast one backend server 808 and a data store 810. It should beunderstood that there can be several backend servers, layers or otherelements, processes or components, which may be chained or otherwiseconfigured, which can interact to perform tasks such as obtaining datafrom an appropriate data store. As used herein, the term “data store”refers to any device or combination of devices capable of storing,accessing and retrieving data, which may include any combination andnumber of data servers, databases, data storage devices and data storagemedia, in any standard, distributed or clustered environment. Thebackend server 808 can include any appropriate hardware and software forintegrating with the data store 810 as needed to execute aspects of oneor more applications for the client device and handling a majority ofthe data access and business logic for an application. The applicationserver provides access control services in cooperation with the datastore and is able to analyze audio date and other data as well asgenerate content such as text, graphics, audio and/or video to betransferred to the user, which may be served to the user by the Webserver 806 in the form of HTML, XML or another appropriate structuredlanguage in this example. The handling of all requests and responses, aswell as the delivery of content between the voice communications device104 and the backend server 808, can be handled by the Web server 806. Itshould be understood that the Web and application servers are notrequired and are merely example components, as structured code discussedherein can be executed on any appropriate device or host machine asdiscussed elsewhere herein. The data store 810 can include severalseparate data tables, databases or other data storage mechanisms andmedia for storing data relating to a particular aspect. For example, thedata store illustrated includes mechanisms for storing content (e.g.,production data) 812 and user information 816, which can be used toserve content for the production side. The data store is also shown toinclude a mechanism for storing log or session data 814. It should beunderstood that there can be other information that may need to bestored in the data store, such as page image information and accessrights information, which can be stored in any of the above listedmechanisms as appropriate or in additional mechanisms in the data store810. The data store 810 is operable, through logic associated therewith,to receive instructions from the backend server 808 and obtain, updateor otherwise process data in response thereto. In one such example, thevoice communications device can receive a request to refine the playbackof media content, such as music, news, audio books, audio broadcasts,and other such content. In this case, the data store might access theuser information to verify the identity of the user and access a mediaservice to determine media content the user is associated with. Theuser's speech can be analyzed and used to generate an updated activeplay queue or initiate the playback of media content. Each servertypically will include an operating system that provides executableprogram instructions for the general administration and operation ofthat server and typically will include computer-readable medium storinginstructions that, when executed by a processor of the server, allow theserver to perform its intended functions. Suitable implementations forthe operating system and general functionality of the servers are knownor commercially available and are readily implemented by persons havingordinary skill in the art, particularly in light of the disclosureherein. The environment in one embodiment is a distributed computingenvironment utilizing several computer systems and components that areinterconnected via communication links, using one or more computernetworks or direct connections. However, it will be appreciated by thoseof ordinary skill in the art that such a system could operate equallywell in a system having fewer or a greater number of components than areillustrated in FIG. 8 . Thus, the depiction of the system 800 in FIG. 8should be taken as being illustrative in nature and not limiting to thescope of the disclosure. The various embodiments can be furtherimplemented in a wide variety of operating environments, which in somecases can include one or more user computers or computing devices whichcan be used to operate any of a number of applications. User or clientdevices can include any of a number of general purpose personalcomputers, such as desktop or laptop computers running a standardoperating system, as well as cellular, wireless and handheld devicesrunning mobile software and capable of supporting a number of networkingand messaging protocols. Such a system can also include a number ofworkstations running any of a variety of commercially-availableoperating systems and other known applications for purposes such asdevelopment and database management. These devices can also includeother electronic devices, such as dummy terminals, thin-clients, gamingsystems and other devices capable of communicating via a network. Mostembodiments utilize at least one network that would be familiar to thoseskilled in the art for supporting communications using any of a varietyof commercially-available protocols, such as TCP/IP, OSI, FTP, UPnP,NFS, CIFS and AppleTalk. The network can be, for example, a local areanetwork, a wide-area network, a virtual private network, the Internet,an intranet, an extranet, a public switched telephone network, aninfrared network, a wireless network and any combination thereof. Inembodiments utilizing a Web server, the Web server can run any of avariety of server or mid-tier applications, including HTTP servers, FTPservers, CGI servers, data servers, Java servers and businessapplication servers. The server(s) may also be capable of executingprograms or scripts in response requests from user devices, such as byexecuting one or more Web applications that may be implemented as one ormore scripts or programs written in any programming language, such asJava, C, C# or C++ or any scripting language, such as Perl, Python orTCL, as well as combinations thereof. The server(s) may also includedatabase servers, including without limitation those commerciallyavailable from Oracle, Microsoft, Sybase and IBM. The environment caninclude a variety of data stores and other memory and storage media asdiscussed above. These can reside in a variety of locations, such as ona storage medium local to (and/or resident in) one or more of thecomputers or remote from any or all of the computers across the network.In a particular set of embodiments, the information may reside in astorage-area network (SAN) familiar to those skilled in the art.Similarly, any necessary files for performing the functions attributedto the computers, servers or other network devices may be stored locallyand/or remotely, as appropriate. Where a system includes computerizeddevices, each such device can include hardware elements that may beelectrically coupled via a bus, the elements including, for example, atleast one central processing unit (CPU), at least one input device(e.g., a mouse, keyboard, controller, touch-sensitive display screen orkeypad, microphone, camera, etc.) and at least one output device (e.g.,a display device, printer or speaker). Such a system may also includeone or more storage devices, such as disk drives, optical storagedevices and solid-state storage devices such as random access memory(RAM) or read-only memory (ROM), as well as removable media devices,memory cards, flash cards, etc. Such devices can also include acomputer-readable storage media reader, a communications device (e.g., amodem, a network card (wireless or wired), an infrared communicationdevice) and working memory as described above. The computer-readablestorage media reader can be connected with, or configured to receive, acomputer-readable storage medium representing remote, local, fixedand/or removable storage devices as well as storage media fortemporarily and/or more permanently containing, storing, sending andretrieving computer-readable information. The system and various devicesalso typically will include a number of software applications, modules,services or other elements located within at least one working memorydevice, including an operating system and application programs such as aclient application or Web browser. It should be appreciated thatalternate embodiments may have numerous variations from that describedabove. For example, customized hardware might also be used and/orparticular elements might be implemented in hardware, software(including portable software, such as applets) or both. Further,connection to other computing devices such as network input/outputdevices may be employed. Storage media and computer readable media forcontaining code, or portions of code, can include any appropriate mediaknown or used in the art, including storage media and communicationmedia, such as but not limited to volatile and non-volatile, removableand non-removable media implemented in any method or technology forstorage and/or transmission of information such as computer readableinstructions, data structures, program modules or other data, includingRAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM,digital versatile disk (DVD) or other optical storage, magneticcassettes, magnetic tape, magnetic disk storage or other magneticstorage devices or any other medium which can be used to store thedesired information and which can be accessed by a system device. Basedon the disclosure and teachings provided herein, a person of ordinaryskill in the art will appreciate other ways and/or methods to implementthe various embodiments.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the invention asset forth in the claims.

What is claimed is:
 1. A system, comprising: at least one computingdevice processor; and a memory device including instructions that, whenexecuted by the at least one computing device processor, cause thesystem to: receive a request for one or more offerings from one or moreelectronic catalogs offered in an e-commerce environment, the requestincluding audio data from a voice-enabled device, the audio datarepresentative of an utterance captured by the device, the deviceassociated with a user account; determine a string of phonemes presentin the utterance; perform automatic speech recognition (ASR) on thestring of phonemes to attempt to interpret the string of phonemes intoone or more words, the automatic speech recognition associated with anindex of words, the index of words associated with the e-commerceenvironment; detect an error in interpreting the string of phonemes, theerror including a low-confidence score or ambiguity between multipleinterpretations; in response to detecting the error, request additionalinformation from the user via the voice-enabled device; receive, inresponse to the request for additional information, the additionalinformation as a voice input, the additional information associated withone or more known words from the index of words; update a user-specificspeech recognition key for the user account to associate the string ofphonemes with the one or more known words; train a general speechrecognition model, associated with the e-commerce environment, using theassociation of the string of phonemes and the one or more known words,wherein the user-specific speech recognition key is updated faster thanthe general speech recognition model is trained.
 2. The system of claim1, wherein the instructions when executed further cause the system to:utilize the additional information to perform subsequent stepsassociated with the request from the voice-enabled device.
 3. The systemof claim 1, wherein the additional information is received in the formof second audio data captured by the voice-enabled device.
 4. The systemof claim 1, wherein the additional information is received in the formof user input entered into a graphical interface displayed on thedisplay-based client device.
 5. A computer-implemented method,comprising: receiving a request for one or more offerings from one ormore electronic catalogs offered in an e-commerce environment, therequest including audio data from a voice-enabled device, the audio datarepresentative of an utterance captured by the device, the deviceassociated with a user account; analyzing the audio data using a generalspeech recognition model and a user-specific speech recognition keyassociated with the user account; determining a string of phonemespresent in the utterance; detecting an error in forming words from thestring of phonemes; receiving, in response to detecting the error, asubsequent user utterance corresponding to the request, the userutterance associated with one or more known words, the one or more knownwords associated with the e-commerce environment; associating the stringof phonemes with the one or more known words; updating the user-specificspeech recognition key with the association of the string of phonemesand the one or more known words; and providing the string of phonemesand the one or more known words to further train the general speechrecognition model, the user-specific speech recognition key beingupdated faster than the general speech recognition model is trained. 6.The method of claim 5, further comprising: requesting additionalinformation from the user via a client device.
 7. The method of claim 6,wherein requesting additional information is performed over voice thougha voice-enabled client device.
 8. The method of claim 6, whereinrequesting additional information is performed via graphical interfaceon a display-based client device.
 9. The method of claim 5, furthercomprising: training a general speech recognition model using theassociation of the string of phonemes and the one or more known words,wherein the general speech recognition model is a statistical model. 10.The method of claim 9, wherein the general speech recognition model isreferenced during speech recognition for a plurality of user accounts,and wherein the user-specific speech recognition key is reference duringspeech recognition for the specific user account.
 11. The method ofclaim 9, wherein the user-specific speech recognition key is updatedfaster than the general speech recognition model.
 12. The method ofclaim 9, further comprising: determining demographic data associatedwith the user account; and training the general speech recognition modelusing an association of the string of phonemes and the one or more knownwords and the demographic data.
 13. The method of claim 9, wherein thegeneral speech recognition model is a part of at least one of anautomatic speech recognition (ASR) model, a natural languageunderstanding (NLU) model, or a named entity recognition (NER) modelassociated with an e-commerce platform.
 14. The method of claim 5,further comprising: receiving a second request including the string ofphonemes; referencing the updated user-specific speech recognition key;and recognizing the string of phonemes as corresponding to the one ormore known words.
 15. The method of claim 9, further comprising:receiving a second request associated with a second user account, thesecond request including a second utterance; determining that the secondutterance includes the string of phonemes; referencing the trainedgeneral speech recognition model; and recognizing the string of phonemesas corresponding to the one or more known words.
 16. A system,comprising: at least one computing device processor; and a memory deviceincluding instructions that, when executed by the at least one computingdevice processor, cause the system to: receive a request for one or moreofferings from one or more electronic catalogs offered in an e-commerceenvironment, the request including audio data from a voice-enableddevice, the audio data representative of an utterance captured by thedevice, the device associated with a user account; analyze the audiodata using a general speech recognition model and a user-specific speechrecognition key associated with the user account; determine a string ofphonemes present in the utterance; detecting an error in forming wordsfrom the string of phonemes; receive, in response to detecting theerror, a subsequent user utterance corresponding to the request, theuser utterance associated with one or more known words, the one or moreknown words associated with the e-commerce environment; associate thestring of phonemes with the one or more known words; update theuser-specific speech recognition key with the association of the stringof phonemes and the one or more known words; and provide the string ofphonemes and the one or more known words to further train the generalspeech recognition model, the user-specific speech recognition key beingupdated faster than the general speech recognition model is trained. 17.The system of claim 16, wherein the instructions that, when executed bythe at least one computing device processor, further cause the systemto: training a general speech recognition model using the association ofthe string of phonemes and the one or more known words, wherein thegeneral speech recognition model is a statistical model.
 18. The systemof claim 17, wherein the general speech recognition model is referencedduring speech recognition for a plurality of user accounts, and whereinthe user-specific speech recognition key is reference during speechrecognition for the specific user account.
 19. The system of claim 17,wherein the user-specific speech recognition key is updated faster thanthe general speech recognition model.
 20. The system of claim 17,wherein the general speech recognition model is a part of at least oneof an automatic speech recognition (ASR) model, a natural languageunderstanding (NLU) model, or a named entity recognition (NER) modelassociated with an e-commerce platform.